Verification of a model of a GUI-based application

ABSTRACT

A method may include receiving a model of a graphical user interface (GUI) based application that includes a plurality of paths. The method may further include determining one or more paths of the plurality of paths that each include a pattern that satisfies a rule-pattern. The rule-pattern may be based on potential inaccuracies in the model as indicated by the pattern. The method may additionally include verifying whether the model is consistent with the GUI-based application. The verification may be based on a prioritization of a determination of whether the one or more paths are consistent with the GUI-based application. The prioritization of the one or more paths may be based on the one or more paths each including the pattern.

FIELD

The embodiments discussed herein are related to verifying a model of agraphical user interface (GUI) based application.

BACKGROUND

Graphical user interface (GUI) based applications for electronicdevices, such as mobile applications for mobile devices, are becomingincreasingly common. Many GUI-based applications execute operationsbased on actions performed on a GUI via a touchscreen, which may easeuse of the GUI-based applications. However, the GUI and touchscreeninterface of the GUI-based applications also provide challenges inassessing the behavior of the GUI-based applications and modeling theGUI-based applications.

The subject matter claimed herein is not limited to embodiments thatsolve any disadvantages or that operate only in environments such asthose described above. Rather, this background is only provided toillustrate one example technology area where some embodiments describedherein may be practiced.

SUMMARY

According to an aspect of an embodiment, a method may include receivinga model of a graphical user interface (GUI) based application thatincludes a plurality of paths. The method may further includedetermining one or more paths of the plurality of paths that eachinclude a pattern that satisfies a rule-pattern. The rule-pattern may bebased on potential inaccuracies in the model as indicated by thepattern. The method may additionally include verifying whether the modelis consistent with the GUI-based application. The verification may bebased on a prioritization of a determination of whether the one or morepaths are consistent with the GUI-based application. The prioritizationof the one or more paths may be based on the one or more paths eachincluding the pattern

The object and advantages of the embodiments will be realized andachieved at least by the elements, features, and combinationsparticularly pointed out in the claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1A illustrates an example system of generating and verifying amodel of a GUI-based application;

FIG. 1B illustrates an example rule-pattern;

FIG. 1C illustrates another example rule-pattern;

FIG. 2A illustrates various screens of a sample GUI-based application;

FIG. 2B illustrates an example DFSM model that may represent thebehavior of the sample GUI-based application of FIG. 2A;

FIG. 3 is a flowchart of an example method of generating a DFSM model ofa GUI-based application;

FIG. 4A is a flowchart of an example method of verifying a DFSM model ofa GUI-based application;

FIG. 4B is an extension of the flowchart of FIG. 4A;

FIG. 5A illustrates an example conjecture model of the sample GUI-basedapplication of FIG. 2A; and

FIG. 5B illustrates another example conjecture model of the sampleGUI-based application of FIG. 2A

DESCRIPTION OF EMBODIMENTS

The behavior of a graphical user interface (GUI) based application(referred to hereinafter as the “GUI-based application”) may be learnedand then modeled as a finite-state model using any number of behaviorlearning techniques and algorithms. For example, a crawler may crawl theGUI-based application and may observe and record behavior of theGUI-based application during the crawl. The recorded behavior may bepassed to a learning engine that may derive a finite-state conjecturemodel (referred hereinafter as “the conjecture model”) of the GUI basedapplication based on the recorded behavior. In some instances, theconjecture model may be inaccurate with respect to the actual behaviorof the GUI-based application.

For example, the crawler may observe and record the behavior of theGUI-based application by performing actions on the GUI of the GUI-basedapplication such that the crawler may observe behavior of the GUI-basedapplication that is directly observable from the GUI of the GUI-basedapplication (referred to hereinafter as “directly observable behavior”).However, the GUI-based application may include behavior that is notdirectly observable on the GUI-based application (referred tohereinafter as “hidden behavior”). Therefore, the crawler may notaccurately observe the true behavior of the GUI-based application suchthat the conjecture model may be inaccurate as compared to the actualbehavior of the GUI-based application.

The hidden behavior of the GUI-based application may manifest as certainpatterns in the directly observable behavior of the GUI-basedapplication. Therefore, paths of the conjecture model that follow thecertain patterns may indicate that hidden behavior of the GUI-basedapplication may be occurring with the directly observable behavior thatmay be represented by the paths that follow the certain patterns.Consequently, the paths that follow the certain patterns may have ahigher likelihood of being inconsistent with the true behavior of theGUI-based application.

As described in detail below, according to some embodiments of thepresent disclosure, the conjecture model may be verified by determiningwhich paths of the conjecture model include a pattern that satisfies arule-pattern. The rule-pattern may be based on potential inaccuracies ofthe conjecture model that may be indicated by the pattern. For example,the rule-pattern may be based on a pattern that is commonly associatedwith hidden behavior of the GUI-based application.

The paths that satisfy the rule-pattern may be prioritized over pathsthat do not satisfy the rule-pattern for determining whether the pathsare consistent with the GUI-based application. The prioritization ofpaths may increase the likelihood of determining early on that theconjecture model is inaccurate and inconsistent with respect to theGUI-based application. Therefore, the efficiency of verifying theconjecture model may be improved as compared to other verifying methodsthat do not perform any rule-based verification. In the presentdisclosure, reference as to whether a model or paths of the model areconsistent, equivalent, or accurate with respect a GUI-based applicationmay refer to whether the model or paths accurately represent the actualbehavior of the GUI-based application.

Embodiments of the present disclosure are explained with reference tothe accompanying drawings.

FIG. 1A illustrates an example system 100 of generating and verifying adeterministic finite state-machine (DFSM) model 106 of a GUI-basedapplication 102, arranged in accordance with at least one embodiment ofthe present disclosure. The GUI-based application 102 may be anysuitable GUI-based application that may be included on an electronicdevice. The electronic device may include, but is not limited to, atablet computer, a smartphone, a navigation system, an entertainmentsystem, a desktop computer, a laptop computer, a digital music player,etc.

The GUI-based application 102 may include one or more states. TheGUI-based application 102 may transition between states based on actionsthat may be performed on the GUI of the GUI-based application 102. Thestates may be associated with a screen of the GUI-based application andmore than one state may be associated with the same screen. When morethan one state is associated with the same screen, differences betweenthe states may be based on hidden behavior of the GUI-based application102 that may not be directly observable from the GUI of the GUI-basedapplication 102. Therefore, in some instances, different states of theGUI-based application 102 may be directly observed on the GUI of theGUI-based application as being the same state. As indicated above anddescribed below, the hidden behavior may create inconsistencies inmodels that may be generated for the GUI-based application 102.

In some embodiments, the system 100 may include a computing system 104.The computing system 104 may include a processor 108 and memory 110. Theprocessor 108 may be any suitable special-purpose or general-purposecomputer, computing entity, or processing device including variouscomputer hardware or software modules and may be configured to executeinstructions stored on any applicable computer-readable media. Forexample, the processor 108 may include a microprocessor, amicrocontroller, a digital signal processor (DSP), anapplication-specific integrated circuit (ASIC), a Field-ProgrammableGate Array (FPGA), or any other digital or analog circuitry configuredto interpret and/or to execute program instructions and/or to processdata. Although illustrated as a single processor in FIG. 1A, it isunderstood that the processor 108 may include any number of processorsconfigured to perform any number of operations together and/orindividually.

The memory 110 may include computer-readable media for carrying orhaving computer-executable instructions or data structures storedthereon. Such computer-readable media may be any available media thatmay be accessed by a general purpose or special purpose computer. By wayof example, and not limitation, such computer-readable media may includetangible or non-transitory computer-readable storage media includingRandom Access Memory (RAM), Read-Only Memory (ROM), ElectricallyErasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-OnlyMemory (CD-ROM) or other optical disk storage, magnetic disk storage orother magnetic storage devices, flash memory devices (e.g., solid statememory devices), or any other storage medium which may be used to carryor store desired program code in the form of computer-executableinstructions or data structures and which may be accessed by a generalpurpose or special purpose computer. Combinations of the above may alsobe included within the scope of computer-readable media.Computer-executable instructions may include, for example, instructionsand data which cause the processor 108 to perform a certain function orgroup of functions.

The computing system 104 may be configured to receive the GUI-basedapplication 102 and may be configured to generate the DFSM model 106 ofthe GUI-based application 102. The DFSM model 106 may represent thebehavior of the GUI-based application 102 and the transitions betweenstates of the GUI-based application. In some embodiments, the computingsystem 104 may be configured to generate the DFSM model 106 according toa method 300 of FIG. 3 described below.

FIG. 2A illustrates various screens of a sample GUI-based application201 (referred to hereinafter as the “sample application 201”) that mayeach be associated with one or more states of the sample application201, according to at least one embodiment described herein. FIG. 2Billustrates an example DFSM model 200 (referred to hereinafter as “themodel 200”) that may represent the behavior of the sample application201, according to at least one embodiment described herein. In someembodiments, the sample application 201 may be an example of theGUI-based application 102 of FIG. 1A and the model 200 may be an exampleof the DFSM model 106 that may be generated by the computing system 104from the sample application 201.

The sample application 201 may be a GUI-based application that helps auser find and locate places nearby from which to obtain food. By way ofexample, and as illustrated in FIG. 2A, the sample application 201 mayinclude a “Home” screen 202, an “About” screen 204, a “Search” screen206, an “Information” screen 208, a “Results” screen 210, and a “Map”screen 212. The “Home” screen 202 may include various widgets that mayallow the user to perform various operations with respect to the sampleapplication 201. The “About” screen 204 may provide information aboutthe sample application 201. The “Search” screen 206 may allow for theuser to search for a place from which to obtain food. The “Information”screen 208 may provide information about a location that may providefood. The “Results” screen 210 may provide the results of a search thatmay be performed on the “Search” screen 206. The “Map” screen 212 mayshow the locations included in the “Results” screen 210 on a map. Therelationships between the different actions that may be performed on thedifferent screens and the associated transitions between states and/orscreens are described in detail below with respect to the model 200.

For example, the model 200 may include a state 220 that may beassociated with the “Home” screen 202, as indicated by the label “Home”in the state 220. The model 200 may include a transition edge 224 fromthe state 220 to a state 230. The state 230 may be associated with the“About” screen 204 as indicated by the label “About” in the state 230.The transition edge 224 may indicate a transition from the state 220 andthe “Home” screen 202 to the state 230 and the “About” screen 204 inresponse to a user tapping an “ABOUT” widget 214 on the “Home” screen202.

The model 200 may include a transition edge 226 from the state 220 to astate 232. The state 232 may be associated with the “Information” screen208 as indicated by the label “Info” in the state 232. The transitionedge 226 may indicate a transition from the state 220 and the “Home”screen 202 to the state 232 and the “Information” screen 208 in responseto a user tapping a “NEAREST” widget 218 on the “Home” screen 202. Themodel 200 may also include a transition edge 228 from the state 232 backto the state 220. The transition edge 228 may indicate a transition fromthe state 232 and the “Information” screen 208 to the state 220 and the“Home” screen 202 in response to a user tapping a “BACK” widget 221 onthe “Information” screen 208 when the sample application 201 is in thestate 232. A self-transition edge 227 of the model 200 indicates thatthe sample application 201 may stay in the state 232 in response to theuser tapping a “TOGGLE” widget 225 on the “Information” screen 208.

Additionally, the model 200 may include a transition edge 229 from thestate 220 to a state 234. The state 234 may be associated with the“Search” screen 206, as indicated by the label “Search” in the state234. The transition edge 229 may indicate a transition from the state220 and the “Home” screen 202 to the state 234 and the “Search” screen206 in response to a user tapping a “SEARCH” widget 216 on the “Home”screen 202. The model 200 may also include a transition edge 231 fromthe state 234 back to the state 220. The transition edge 231 mayindicate a transition from the state 234 and the “Search” screen 206 tothe state 220 and the “Home” screen 202 in response to a user tapping a“BACK” widget 223 on the “Search” screen 206 when the sample application201 is in the state 234.

The model 200 may also include a transition edge 236 from the state 234to a state 238. The state 238 may be associated with the “Results”screen 210 as indicated by the label “Results” in the state 238. Thetransition edge 236 may indicate a transition from the state 234 and the“Search” screen 206 to the state 238 and the “Results” screen 210 inresponse to a user tapping a “FIND” widget 242 on the “Search” screen206. The model 200 may also include a transition edge 240 from the state238 back to the state 234. The transition edge 240 may indicate atransition from the state 238 and the “Results” screen 210 back to thestate 234 and the “Search” screen 206 in response to a user tapping a“BACK” widget 244 on the “Results” screen 210 when the sampleapplication 201 is in the state 238.

The model 200 may also include a transition edge 248 from the state 238to a state 246. The state 246 may be associated with the “Map” screen212 as indicated by the label “Map” in the state 246. The transitionedge 248 may indicate a transition from the state 238 and the “Results”screen 210 to the state 246 and the “Map” screen 212 in response to auser tapping a “MAP” widget 252 on the “Results” screen 210. The model200 may also include a transition edge 250 from the state 246 back tothe state 238. The transition edge 250 may indicate a transition fromthe state 246 and the “Map” screen 212 back to the state 238 and the“Results” screen 210 in response to a user tapping a “BACK” widget 254on the “Map” screen 212 when the sample application 201 is in the state246.

The model 200 may also include a transition edge 258 from the state 238to a state 256. The state 256 may be associated with the “Information”screen 208 as indicated by the label “Info” in the state 256. Thetransition edge 258 may indicate a transition from the state 238 and the“Results” screen 210 to the state 256 and the “Information” screen 208in response to a user tapping a “LOCATION” widget 262 on the “Results”screen 210. The model 200 may also include a transition edge 260 fromthe state 256 to the state 238. The transition edge 260 may indicate atransition from the state 256 and the “Information” screen 208 to thestate 238 and the “Results” screen 210 in response to a user tapping the“BACK” widget 221 on the “Information” screen 208 when the sampleapplication 201 is in the state 256. A self-transition edge 264 of themodel 200 indicates that the sample application 201 may stay in thestate 256 in response to the user tapping the “TOGGLE” widget 225 on the“Information” screen 208.

The sample application 201 and the model 200 depicted in FIGS. 2A and 2Bare merely examples and are not meant to be limiting. Any number of DFSMmodels may be generated for any number of GUI-based applications inconcordance with the present disclosure.

Returning to FIG. 1A, in some embodiments, during the generation of theDFSM model 106, the computing system 104 may generate a conjecture DFSMmodel (“conjecture model”). The computing system 104 may be configuredto verify the conjecture model to determine whether the conjecture modelis substantially consistent with the GUI-based application 102. When theconjecture model is deemed to be substantially consistent with theGUI-based application 102, the conjecture model may be output as theDFSM model 106. In some embodiments, the computing system 104 may beconfigured to verify the conjecture model according to a method 400 ofFIGS. 4A and 4B, described below.

In some embodiments, the computing system 104 may be configured toverify the conjecture model by determining whether one or more paths ofthe conjecture model are consistent with the GUI-based application 102.Determining whether a path is consistent with the GUI-based application102 may also be referred to as verifying the path. In these or otherembodiments, paths of the conjecture model may be ordered according to aprioritization scheme. The prioritization scheme may dictate which pathsare verified first.

In some embodiments, the prioritization of the paths may be based onwhich paths include patterns that satisfy one or more rule-patterns. Therule-patterns may be based on potential inaccuracies in the conjecturemodels that may be indicated by the patterns. As indicated above,inaccuracies in the conjecture model may be due to hidden behavior inthe GUI-based application 102. Therefore, in some embodiments, therule-patterns may be based on potential patterns in topologies of theconjecture model that may indicate hidden behavior of the GUI-basedapplication 102. In these or other embodiments, one or more of therule-patterns may indicate instances when multiple states of theGUI-based application 102 are represented by a single state in theconjecture model.

FIG. 1B illustrates an example rule-pattern 111, according to at leastone embodiment described herein. The rule-pattern 111 may include apattern that may potentially manifest in the topology of the conjecturemodel and that may, but does not necessarily, indicate hidden behaviorof the GUI-based application 102. Therefore, in some instances, therule-pattern 111 may indicate an inconsistency between the conjecturemodel and the GUI-based application 102. The rule-pattern 111 mayinclude a first state 112, a second state 114, and a third state 116.The rule-pattern 111 may follow a sequence of actions “‘a’→‘b’→‘c’” inwhich a transition may occur from the first state 112 to the secondstate 114 in response to the action “a,” a transition may occur from thesecond state 114 to the third state 116 in response to the action “b,”and a transition may occur from the third state 116 back to the secondstate 114 in response to the action “c.”

In some instances, the rule-pattern 111 may indicate hidden behavior ofthe GUI-based application 102 that may be associated with a “back”operation of the GUI-based application 102. For example, the GUI-basedapplication 102 may transition to a particular screen of the GUI-basedapplication 102 from two or more different screens of the GUI-basedapplication 102 such that two or more different states may be associatedwith the particular screen. Therefore, when a “back” operation isperformed on the particular screen with respect to a particular state,the GUI-based application 102 may transition to different screens andstates depending on from which screen and state the transition to theparticular screen and the particular state occurred. The determinationas to which screen and state to transition to from the particular screenand state in response to the “back” operation may include behavior thatis hidden from the GUI of the GUI-based application 102.

Because of the hidden behavior associated with the “back” operation, theconjecture model may represent multiple states of the GUI-basedapplication 102 as a single state such that the conjecture model mayinaccurately represent the behavior associated with the “back”operation. For example, the conjecture model may include therule-pattern 111 where the action “b” in the sequence of actions“‘a’→‘b’→‘c’” may be associated with a “back” operation such that thesecond state 114 may represent more than one state in the GUI-basedapplication 102. Therefore, the conjecture model may inaccuratelyrepresent that the GUI-based application 102 may transition from thesecond state 114 to the third state 116 in response to the “back”operation instead of transitioning back to the first state 112 inresponse to the “back” operation.

FIG. 1C illustrates an example rule-pattern 118, according to at leastone embodiment described herein. Like the rule-pattern 111 of FIG. 1B,the rule-pattern 118 may include a pattern that may potentially manifestin the topology of the conjecture model and that may, but does notnecessarily, indicate hidden behavior of the GUI-based application 102.Therefore, the rule-pattern 118 may also indicate an inconsistencybetween the conjecture model and the GUI-based application 102.

The rule-pattern 118 may include a first state 120, a second state 122,a third state 124, and a fourth state 126. The second state 122 and thethird state 124 may include the same potential actions, e.g., the secondstate 122 and the third state 124 may be associated with the same screenof the GUI-based application 102. The rule-pattern 118 may follow asequence of actions “‘a’→‘c’→‘c’” in which a transition may occur fromthe first state 120 to the second state 122 in response to the action“a,” a transition may occur from the second state 122 to the third state124 in response to the action “c,” and a transition may occur from thethird state 124 to the fourth state 126 in response to the action “e.”

The rule-pattern 118 may represent a potential inconsistency where ascreen of the corresponding GUI-based application may be represented bythe states 122 and 124, and where the screen supports the action “c”which may cause the GUI-based application to remain in the same screenand the same state in a self-loop. However, in some instances, thisbehavior may be incorrectly represented in a conjecture model as atransition from one state associated with the screen (e.g., the state122) to another state associated with the screen (e.g., the state 124)such as illustrated in the rule-pattern 118. In contrast, in aconsistent and correct model of this behavior, each of the two statesassociated with the same screen may have a self-loop associated with theaction “c”.

Returning to FIG. 1A, in some embodiments, the computing system 104 maybe configured to sort the rule-patterns based on specificity. Thespecificity may relate to how likely a path that follows therule-pattern is inconsistent with the GUI-based application 102. Forexample, a higher percentage of paths that follow the rule-pattern 111of FIG. 1B are likely to be consistent with the GUI-based application102 than a percentage of paths that follow the rule-pattern 118 of FIG.1C. Therefore, the rule-pattern 118 may be considered as being morespecific than the rule-pattern 111.

The computing system 104 may be configured to verify the paths of theconjecture model based on the ordered specificity of the rule-patterns.For example, the computing system 104 may be configured to determinewhich paths of the conjecture model satisfy the rule-pattern 118 and toverify those paths before determining which paths satisfy therule-pattern 111 and verifying those corresponding paths. Therefore, thelikelihood of determining an inconsistency early on may be increased.

In these or other embodiments, the computing system 104 may beconfigured to determine one or more priority weights for one or morepaths of the conjecture model. In some embodiments, the priority weightsmay be based on one or more heuristics that may be applied to the paths.The heuristics may be based on properties of the paths that may alsoindicate that a potential inconsistency may occur.

For example, a large number of incoming edges associated with a state inthe conjecture model may indicate that the state is representingmultiple states of the GUI-based application 102. Therefore, in someembodiments, the heuristics may include a number of incoming edgesassociated with one or more states of the paths. The paths with thehigher number of incoming edges associated with one or morecorresponding states may be given a higher priority weight.

The computing system 104 may be configured to verify the paths of theconjecture model based on the priority weights also. For example, thecomputing system 104 may be configured to verify paths having a higherpriority weight before verifying paths having a lower priority weight.In some embodiments the paths that satisfy a particular rule-pattern maybe verified in an order that is based on their respective priorityweights.

Accordingly, the system 100 may be configured to generate and verify aDFSM model of a GUI-based application. Modifications, additions, oromissions may be made to the system 100 without departing from the scopeof the present disclosure. For example, the rule-patterns 111 and 118are merely examples of rule-patterns that may be used to verify aconjecture model. Other rule-patterns may also be used to verify theconjecture model. Further, the number of incoming edges is merely anexample heuristic that may be used for determining a priority weight forthe paths. For example the length of a path may be used as a criterionfor prioritization. Short paths may not cover substantial enoughbehavior to expose inconsistencies, while really long paths may be toomany in number and may be relatively expensive to verify. Thus, anon-monotonic prioritization scheme which favors medium-length paths maybe used.

Another heuristic could be to prioritize paths based on the length ofthe longest sub-sequence of that path that has been previously explored.The smaller this sub-sequence, the more likely this path may exploresubstantially new behavior and may thus be more likely to expose aninconsistency. Yet another heuristic may be to prioritize pathscontaining one or more instances of actions from a pre-defined set ofactions that are often implicated in inconsistent behavior. The “BACK”action is an example of such an action.

FIG. 3 is a flowchart of an example method 300 of generating a DFSMmodel of a GUI-based application, according to at least one embodimentof the present disclosure. The method 300 may be implemented in someembodiments by a computing system, such as the computing system 104 ofFIG. 1A. For instance, the processor 108 of the computing system 104 maybe configured to execute computer instructions stored in the memory 110to perform actions for generating the DFSM model 106 of the GUI-basedapplication 102, as represented by one or more of blocks of the method300. Although illustrated as discrete blocks, various blocks may bedivided into additional blocks, combined into fewer blocks, oreliminated, depending on the desired implementation.

The method 300 may begin, and at block 302, where a GUI-basedapplication “app” may be received. For example, the GUI-basedapplication 102 of FIG. 1A may be received at block 302. At block 304,“app” may be crawled to observe and extract behavior “b” of “app.” Thecrawling may be performed by any suitable crawler (e.g., a dynamiccrawler) that may automatically interact with “app” through the GUI of“app” such that the crawler may record and observe the behavior “b” of“app,” as indicated by changes in the GUI of “app.”

At block 306, a conjecture model “model” of “app” may be learned fromthe behavior “b.” The learning may be performed by any suitable learningengine, which may be a passive learning engine or an active learningengine. A passive learning engine may generate “model” based on thebehavior “b” that may be observed by the crawler without any input fromthe passive leaning engine. In contrast, an active learning engine maydirect the crawler to crawl one or more portions of “app” in generatingthe behavior “b” such that the active learning engine may provide inputcorresponding to the extraction of the behavior “b.”

At block 308, it may be determined whether “model” is substantiallyequivalent to “app.” As indicated above, a determination as to whether“model” is substantially equivalent to “app” may be a determination asto whether “model” accurately represents the behavior of “app.” In someembodiments, the determination performed at block 308 may be performedaccording to a method 400 described below with respect to FIGS. 4A and4B. When “model” is not substantially equivalent to “app,” aninconsistency “p” between “model” and “app” may be observed and themethod 300 may proceed from block 308 to block 310. In some embodiments,the inconsistency “p” may be identified according to one or moreoperations associated with the method 400 also. When “model” issubstantially equivalent to “app,” the method 300 may proceed from block308 to block 314.

At block 310, the inconsistency “p” may be retrieved. At block 312, thebehavior “b” may be augmented with respect to the inconsistency “p.” Forexample, in some embodiments, the inconsistency “p” may be a path of“app” that is inconsistent with a path in “model.” In these and otherembodiments, the inconsistency “p” may then be used as a counter-examplethat may be fed to the crawler. The crawler may then crawl “app” basedon the inconsistency “p” such that additional behavior of “app” may beobserved and extracted. The additional behavior may be added to thebehavior “b” such that the behavior “b” may in effect be augmented withrespect to the inconsistency “p.” Following block 312, the method 300may return to block 306, where a new conjecture model “model” may belearned based on the augmented behavior “b.”

At block 314, when “model” is substantially equivalent to “app,” “model”may be output as a verified DFSM model of “app.” For example, withrespect to FIG. 1A, the DFSM model 106 may be output at block 314 as averified model of the GUI-based application 102.

Therefore, the method 300 may be used to generate a DFSM model of aGUI-based application. Modifications, additions, or omissions may bemade to the method 300 without departing from the scope of the presentdisclosure. For example, one skilled in the art will appreciate that forthe method 300, the functions performed may be implemented in differingorder. Furthermore, the outlined steps and actions are only provided asexamples, and some of the steps and actions may be optional, combinedinto fewer steps and actions, or expanded into additional steps andactions without detracting from the essence of the disclosedembodiments. For example, the operations performed at block 308 mayinclude any number of additional operations, such as those describedbelow with respect to FIGS. 4A and 4B.

FIGS. 4A and 4B illustrate a flowchart of an example method 400 ofverifying a DFSM model of a GUI-based application, arranged inaccordance with at least one embodiment of the present disclosure. Asmentioned above, the method 400 may be used to perform one or moreactions associated with blocks 308 and 310 of the method 300 describedabove. The method 400 may be implemented in some embodiments by acomputing system, such as the computing system 104 of FIG. 1. Forinstance, the processor 108 of the computing system 104 may beconfigured to execute computer instructions stored in the memory 110 toperform actions verifying a DFSM model of the GUI-based application 102,as represented by one or more of blocks of the method 400. Althoughillustrated as discrete blocks, various blocks may be divided intoadditional blocks, combined into fewer blocks, or eliminated, dependingon the desired implementation.

The method 400 may begin, and at block 402 a GUI-based application “app”may be received. For example, the GUI-based application 102 of FIG. 1may be retrieved. As indicated above, in some embodiments the GUI-basedapplication “app” of FIGS. 4A and 4B may include the same GUI-basedapplication “app” of FIG. 3.

At block 404, a conjecture model “model” may be received. As indicatedabove, the conjecture model “model” may include the conjecture model“model” that may be learned at block 306 of FIG. 3.

At block 406, a set of rule-patterns “rules” may be received. The set ofrule-patterns “rules” may include one or more patterns that may indicatepotential inaccuracies in the conjecture model “model.” For example, theset of set of rule-patterns “rules” may include the rule-patterns 111and 118 described above with respect to FIGS. 1B and 1C, respectively.Additionally, in some embodiments, the set of rule-patterns “rules” maybe sorted according to specificity.

At block 408, a set of unexecuted paths “pathSet” of the conjecturemodel “model” may be received. In the present disclosure, a path may beconsidered “unexecuted” in the context of the method 400 when that pathhas not yet been verified. For example, paths previously executed on“app” and directly used to learn the conjecture model “model” may beconsidered as verified. At block 410, priority weights may be determinedfor the paths in “pathSet.” In some embodiments, the priority weightsmay be determined based on one or more heuristics, as described above.For example, the priority weights may be determined based on the numberof incoming edges to one or more states of the paths in “pathSet.”

At block 412, it may be determined whether there are any unappliedrule-patterns in the set of rule-patterns “rules.” In the presentdisclosure, a particular rule-pattern may be considered “unapplied” inthe context of the method 400 when it has not yet been determinedwhether one or more paths of the conjecture model “model” include theparticular rule-pattern. When the set of rule-patterns “rules” includesone or more unapplied rule-patterns, the method 400 may proceed to block414.

At block 414, a next unapplied rule-pattern “r” may be retrieved fromthe set of rule-patterns “rules.” At block 416, it may be determinedwhether any paths in “pathSet” match the rule-pattern “r.” In thepresent disclosure a path may be considered as “matching” or“satisfying” a rule-pattern when the path includes the pattern thatmatches the rule-pattern. When one or more paths in “pathSet” match therule-pattern “r,” the method 400 may proceed to block 418. When no pathsin “pathSet” match the rule-pattern “r,” the method 400 may return toblock 412.

At block 418, the paths from “pathSet” that match the rule-pattern “r”(“matches”) may be received. At block 420, the paths included in“matches” may be sorted according to their priority weights. At block422, it may be determined whether “matches” includes any unexecutedpaths. When “matches” includes one or more unexecuted paths, the method400 may proceed to block 424. When “matches” does not include one ormore unexecuted paths, the method 400 may return to block 412.

At block 424, a next unexecuted path “p” from “matches” may be selected.At block 426, it may be determined whether path “p” is consistent with“app.” When path “p” is not consistent with “app,” the method 400 mayproceed to block 428. At block 428, the path “p” may be output as aninconsistency with respect to “app.” In some embodiments, the path “p”may then function as the inconsistency “p” of blocks 310 and 312 of themethod 300 described above.

When path “p” is consistent with “app” at block 426, method 400 mayreturn to block 422. One or more of blocks 412, 414, 416, 418, 420, 422,424, and 426 may be repeated until an inconsistency is found at block426 or until it is determined at block 412 that there are no moreunapplied rule-patterns in “rules.” As indicated above, when the set ofrule-patterns “rules” does not include one or more unappliedrule-patterns, the method 400 may proceed to block 430. When the method400 reaches block 430 after applying the rule-patterns in “rules,” theconjecture model “model” may be considered as being substantiallyequivalent to “app.” As such, at block 430, it may be determined thatthe conjecture model “model” is substantially equivalent to “app.” Asindicated above with respect to block 314 of the method 300, in someembodiments, the conjecture model “model” may be output as a DFSM modelfor the GUI-based application “app” when it is determined at block 430that the conjecture model “model” is substantially equivalent to “app.”

Modifications, additions, or omissions may be made to the method 400without departing from the scope of the present disclosure. For example,one skilled in the art will appreciate that for the method 400, thefunctions performed may be implemented in differing order. Furthermore,the outlined steps and actions are only provided as examples, and someof the steps and actions may be optional, combined into fewer steps andactions, or expanded into additional steps and actions withoutdetracting from the essence of the disclosed embodiments. For example,in some embodiments, after a particular number of paths that satisfy aparticular rule-pattern have been verified and no inconsistencies havebeen found via blocks 422, 424, and 426, the method 400 may move ontoblock 412 even if all the paths that satisfy the particular rule-patternhave not been verified.

As indicated above, the methods 300 and 400 may be used to generate andverify DFSM models for a GUI-based application. For example, FIG. 5Aillustrates an example conjecture model 500 a of the sample application201 of FIG. 2A. The conjecture model 500 may be generated using blocks302, 304, and 306 of the method 300. Additionally, one or more blocks ofthe method 400 may be used to verify whether the conjecture model 500 issubstantially equivalent to the sample application 201.

For example, in some embodiments, the computing system 104 of FIG. 1Amay receive the sample application 201, the conjecture model 500 a, anda set of rule-patterns per blocks 402, 404, and 406 of the method 400.The set of rule-patterns may include the rule-patterns 111 and 118 ofFIGS. 1B and 1C respectively. Additionally, the set of rule-patterns maybe ordered based on specificity in which the rule-pattern 118 may beprioritized over the rule-pattern 111 because the rule-pattern 118 maybe more specific than the rule-pattern 111, as described above.

Additionally, per blocks 408 and 410 of the method 400, the computingsystem 104 may determine the unexecuted paths of the conjecture model500 a and may compute priority weights for the unexecuted paths of theconjecture model 500 a. In the illustrated example, the priority weightsmay be determined based on the number of incoming edges with respect tothe states of the unexecuted paths.

Per blocks 412, 414, and 416, the computing system 104 may firstretrieve the rule-pattern 118 from the set of rule-patterns and maydetermine whether any paths in the conjecture model 500 a match therule-pattern 118. In the illustrated example, there may not be any pathsin the conjecture model 500 a that match the rule-pattern 118.Therefore, the computing system may then retrieve the rule-pattern 111from the set of rule-patterns and may determine whether any paths of theconjecture model match the rule-pattern 111. In the illustratedembodiment, the conjecture model 500 a may include 34 paths that matchthe rule-pattern 111. The computing system 104 may attempt to apply therule-patterns 118 and 111 in the order indicated based on theprioritization of the rule-patterns 118 and 111.

Per blocks 418 and 420, the computing system 104 may retrieve the pathsthat match the rule-pattern 111 and may order them according to theirpriority weights. For example, Table 1 below includes the actionsequences of the top 10 (according to highest priority weight) of the 34paths that match the rule-pattern 111.

TABLE 1 1. “LOCATIONS” → “BACK” → “NEAREST” → “TOGGLE” → “TOGGLE” 2.“BACK” → “SEARCH” → “BACK” → “NEAREST” → “TOGGLE” 3. “BACK” → “NEAREST”→ “BACK” → “NEAREST” → “TOGGLE” 4. “LOCATIONS” → “BACK” → “NEAREST” →“TOGGLE” → “BACK” 5. “LOCATIONS” → “BACK” → “NEAREST” → “BACK” →“NEAREST” 6. “BACK” → “SEARCH” → “BACK” → “NEAREST” → “BACK” 7. “BACK” →“NEAREST” → “BACK” → “NEAREST” → “BACK” 8. “LOCATIONS” → “BACK” →“NEAREST” → “BACK” → “SEARCH” 9. “SEARCH” → “FIND” → “BACK” → “BACK” →“NEAREST” 10. “BACK” → “SEARCH” → “BACK” → “SEARCH” → “FIND”

Per blocks 422 and 424, the computing system 104 may first select path“1” indicated in Table 1 to determine whether path “1” is consistentwith the sample application 201 based on the ordering of the paths,which may be based on the priority weight. Per block 426, the computingsystem 104 may determine whether path “1” is inconsistent with thesample application 201. In the illustrated example, path “1” may beinconsistent with the sample application 201. Therefore, the computingsystem 104 may output path “1” as an inconsistency with respect to thesample application 201.

Based on the inconsistency associated with path “1,” in someembodiments, the computing system 104 may return to method 300 and maygenerate a new conjecture model per blocks 310, 312, and 306. FIG. 5Billustrates an example conjecture model 500 b of the sample application201 of FIG. 2B that may be generated based on the inconsistencyassociated with path “1.”

The computing system 104 may then return to the method 400 and mayreceive the sample application 201, the conjecture model 500 b, and theset of rule-patterns per blocks 402, 404, and 406 of the method 400. Aswith application of the method 400 with respect to the conjecture model500 a, the set of rule-patterns may include the rule-patterns 111 and118 of FIGS. 1B and 1C, respectively. Additionally, the set ofrule-patterns may be ordered based on specificity in which therule-pattern 118 may be prioritized over the rule-pattern 111 becausethe rule-pattern 118 may be more specific than the rule-pattern 111, asdescribed above.

Per blocks 408 and 410 of the method 400, the computing system 104 maydetermine the unexecuted paths of the conjecture model 500 b and maycompute priority weights for the unexecuted paths of the conjecturemodel 500 b. In the illustrated example, the priority weights may bedetermined based on the number of incoming edges with respect to thestates of the unexecuted paths.

Per blocks 412, 414, and 416, the computing system 104 may firstretrieve the rule-pattern 118 from the set of rule-patterns and maydetermine whether any paths in the conjecture model 500 b match therule-pattern 118. In the illustrated example, a path 503 of theconjecture model 500 b that follows the action sequence“‘NEAREST’→‘TOGGLE’→‘BACK’” may match the rule-pattern 118. Per block418, the computing system 104 may retrieve the path 503 that matches therule-pattern 118. In the illustrated example, block 420 may be skipped,because the path 503 may be the only path that follows the rule-pattern118.

Per blocks, 422, 424, and 426, the computing system 104 may determinewhether the path 503 is inconsistent with the sample application 201. Inthe illustrated example, the path 503 may be inconsistent with thesample application 201. Therefore, the computing system 104 may outputthe path 503 as an inconsistency with respect to the sample application201.

Based on the inconsistency associated with the path 503, in someembodiments, the computing system 104 may return to method 300 and maygenerate a new conjecture model per blocks 310, 312, and 306. In someembodiments, the computing system 104 may output the model 200 of FIG.2A as the conjecture model. The computing system 104 may then apply themethod 400 to the model 200 and may determine that the model 200 issubstantially equivalent to the sample application 201. As such, thecomputing system 104 may output the model 200 as a DFSM model of thesample application 201.

The above description with respect to the sample application is merelyan example and is not meant to be limiting. Additionally, the order ofsteps performed is merely exemplary. The steps and processes involvedmay be performed in differing orders and may include more or fewersteps.

One skilled in the art will appreciate that, for these and otherprocesses and methods disclosed herein, the functions performed in theprocesses and methods may be implemented in differing order.Furthermore, the outlined steps and actions are only provided asexamples, and some of the steps and actions may be optional, combinedinto fewer steps and actions, or expanded into additional steps andactions without detracting from the essence of the disclosedembodiments.

As indicated above, the embodiments described herein may include the useof a special purpose or general purpose computer (e.g., the processor108 of FIG. 1) including various computer hardware or software modules,as discussed in greater detail below.

Further, as indicated above, embodiments described herein may beimplemented using computer-readable media (e.g., the memory 110 ofFIG. 1) for carrying or having computer-executable instructions or datastructures stored thereon. Such computer-readable media may be anyavailable media that may be accessed by a general purpose or specialpurpose computer. By way of example, and not limitation, suchcomputer-readable media may include tangible or non-transitorycomputer-readable storage media including RAM, ROM, EEPROM, CD-ROM orother optical disk storage, magnetic disk storage or other magneticstorage devices, flash memory devices (e.g., solid state memorydevices), or any other storage medium which may be used to carry orstore desired program code in the form of computer-executableinstructions or data structures and which may be accessed by a generalpurpose or special purpose computer. Combinations of the above may alsobe included within the scope of computer-readable media.

Computer-executable instructions may include, for example, instructionsand data which cause a general purpose computer, special purposecomputer, or special purpose processing device (e.g., one or moreprocessors) to perform a certain function or group of functions.Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

As used herein, the terms “module” or “component” may refer to specifichardware implementations configured to perform the actions of the moduleor component and/or software objects or software routines that may bestored on and/or executed by general purpose hardware (e.g.,computer-readable media, processing devices, etc.) of the computingsystem. In some embodiments, the different components, modules, engines,and services described herein may be implemented as objects or processesthat execute on the computing system (e.g., as separate threads). Whilesome of the system and methods described herein are generally describedas being implemented in software (stored on and/or executed by generalpurpose hardware), specific hardware implementations or a combination ofsoftware and specific hardware implementations are also possible andcontemplated. In this description, a “computing entity” may be anycomputing system as previously defined herein, or any module orcombination of modulates running on a computing system.

All examples and conditional language recited herein are intended forpedagogical objects to aid the reader in understanding the invention andthe concepts contributed by the inventor to furthering the art, and areto be construed as being without limitation to such specifically recitedexamples and conditions. Although embodiments of the present disclosurehave been described in detail, it should be understood that the variouschanges, substitutions, and alterations could be made hereto withoutdeparting from the spirit and scope of the present disclosure.

What is claimed is:
 1. A method comprising: receiving a model of agraphical user interface (GUI) based application, the model include s aplurality of states that represent states of the GUI-based applicationand the model includes a plurality of paths that indicate observedbehavior of the GUI-based application in which each of the plurality ofpaths includes one or more transitions between two or more states of themodel that indicate a sequence of one or more actions performed on theGUI-based application that result in the one or more transitions;determining that one or more paths of the plurality of paths of themodel each include a particular series of transitions between states ofthe model with a pattern that satisfies a rule-pattern in whichsatisfaction of the rule-pattern indicates potential inconsistencies inthe model with respect to actual behavior of the GUI-based application,wherein the potential inconsistencies are based on at least one ofhidden behavior present in the one or more paths and multiple states ofthe GUI-based application being represented by a single state in themodel; determining an order of the plurality of paths for determiningwhether the one or more paths are consistent with the GUI-basedapplication over other paths of the GUI-based application by applying apriority weight to the one or more paths, wherein a higher priorityweight for the order is given to the one or more paths over the otherpaths based on the one or more paths including the pattern thatsatisfies the rule-pattern and the higher priority weight is given tothe one or more paths over the other paths based on a number of incomingedges associated with one or more states of the one or more paths beinggreater than the number of incoming edges associated with one or morestates of the other paths; and verifying whether the model is consistentwith the GUI-based application based on the determined order of theplurality of paths.
 2. The method of claim 1, wherein the one or morepaths are one or more first paths, the pattern is a first pattern, andthe rule-pattern is a first rule-pattern, and the method furthercomprises: determining that one or more second paths of the plurality ofpaths of the model each include a second pattern that satisfies a secondrule-pattern, the second rule-pattern is based on the second patternindicating potential inconsistencies in the model with respect to actualbehavior of the GUI-based application; and verifying whether the modelis consistent with the GUI-based application based on a prioritizationof determining whether the one or more second paths are consistent withthe GUI-based application, the prioritization of the one or more secondpaths is based on determining that the one or more second paths eachinclude the second pattern that satisfies the second rule-pattern. 3.The method of claim 2, further comprising prioritizing determiningwhether the one or more second paths are consistent with the GUI-basedapplication over determining whether the one or more first paths areconsistent with the GUI-based application.
 4. The method of claim 3,wherein prioritizing determining whether the one or more second pathsare consistent with the GUI-based application over determining whetherthe one or more first paths are consistent with the GUI-basedapplication is based on the second rule-pattern being more specific thanthe first rule-pattern.
 5. The method of claim 1, wherein therule-pattern includes a particular sequence of actions that results in afirst transition from a first state of the model to a second state ofthe model followed by a second transition from the second state to athird state of the model followed by a third transition from the thirdstate to a fourth state of the model where the second state and thethird state include the same potential actions.
 6. The method of claim1, wherein the rule-pattern includes a particular sequence of actionsthat results in a first transition from a first state of the model to asecond state of the model followed by a second transition from thesecond state to a third state of the model followed by a thirdtransition from the third state back to the second state.
 7. One or morenon-transitory computer-readable storage media including computerexecutable instructions configured to cause a system to performoperations to verify a model of a graphical user interface (GUI) basedapplication, the operations comprising: receiving a model of a graphicaluser interface (GUI) based application, the model include s a pluralityof states that represent states of the GUI-based application and themodel includes a plurality of paths that indicate observed behavior ofthe GUI-based application in which each of the plurality of pathsincludes one or more transitions between two or more states of the modelthat indicate a sequence of one or more actions performed on theGUI-based application that result in the one or more transitions;determining that one or more paths of the plurality of paths of themodel each include a particular series of transitions between states ofthe model with a pattern that satisfies a rule-pattern in whichsatisfaction of the rule-pattern indicates potential inconsistencies inthe model with respect to actual behavior of the GUI-based application,wherein the potential inconsistencies are based on at least one ofhidden behavior present in the one or more paths and multiple states ofthe GUI-based application being represented by a single state in themodel; determining an order of the plurality of paths for determiningwhether the one or more paths are consistent with the GUI-basedapplication over other paths of the GUI-based application by applying apriority weight to the one or more paths, wherein a higher priorityweight for the order is given to the one or more paths over the otherpaths based on the one or more paths including the pattern thatsatisfies the rule-pattern and the higher priority weight is given tothe one or more paths over the other paths based on a number of incomingedges associated with one or more states of the one or more paths beinggreater than the number of incoming edges associated with one or morestates of the other paths; and verifying whether the model is consistentwith the GUI-based application based on the determined order of theplurality of paths.
 8. The one or more non-transitory computer-readablestorage media of claim 7, wherein the one or more paths are one or morefirst paths, the pattern is a first pattern, and the rule-pattern is afirst rule-pattern, and the operations further comprise: identifying oneor more second paths of the plurality of paths of the model that eachinclude a second pattern that satisfies a second rule-pattern, thesecond rule-pattern is based on the second pattern indicating potentialinconsistencies in the model with respect to actual behavior of theGUI-based application; and verifying whether the model is consistentwith the GUI-based application based on a prioritization of determiningwhether the one or more second paths are consistent with the GUI-basedapplication, the prioritization of the one or more second paths based onthe one or more second paths each include the second pattern thatsatisfies the second rule-pattern.
 9. The one or more non-transitorycomputer-readable storage media of claim 8, wherein the operationsfurther comprise prioritizing determining whether the one or more secondpaths are consistent with the GUI-based application over determiningwhether the one or more first paths are consistent with the GUI-basedapplication.
 10. The one or more non-transitory computer-readablestorage media of claim 9, wherein prioritizing determining whether theone or more second paths are consistent with the GUI-based applicationover determining whether the one or more first paths are consistent withthe GUI-based application is based on the second rule-pattern being morespecific than the first rule-pattern.
 11. The one or more non-transitorycomputer-readable storage media of claim 7, wherein the rule-patternincludes a particular sequence of actions that results in a firsttransition from a first state of the model to a second state of themodel followed by a second transition from the second state to a thirdstate of the model followed by a third transition from the third stateto a fourth state of the model where the second state and the thirdstate include the same potential actions.
 12. The one or morenon-transitory computer-readable storage media of claim 7, wherein therule-pattern includes a particular sequence of actions that results in afirst transition from a first state of the model to a second state ofthe model followed by a second transition from the second state to athird state of the model followed by a third transition from the thirdstate back to the second state.